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Abstract 

The fusion of the multi-agent paradigm with evolutionary computation yielded 
promising results in many optimization problems. Evolutionary multi-agent sys¬ 
tem (EMAS) are more similar to biological evolution than classical evolutionary 
algorithms. However, technological limitations prevented the use of fully asyn¬ 
chronous agents in previous EMAS implementations. In this paper we present a 
new algorithm for agent-based evolutionary computations. The individuals are 
represented as fully autonomous and asynchronous agents. An efficient imple¬ 
mentation of this algorithm was possible through the use of modern technologies 
based on functional languages (namely Erlang and Scala), which natively sup¬ 
port lightweight processes and asynchronous communication. Our experiments 
show that such an asynchronous approach is both faster and more efficient in 
solving common optimization problems. 

Keywords: Multi-agent systems, Evolutionary computing. Functional 
programming 


1. Introduction 

Biological systems are asynchronous by nature. This fact is not always con¬ 
sidered in biologically-inspired computing methods (e.g. metaheuristics, such 
as evolutionary algorithms [21]). These systems usually use notions such as 
“discrete” generations, loosing such concepts as parallel ontogenesis, or lack of 
global control. Nevertheless, such computing systems have proven to be effective 
in different optimization problems. Moreover, some of them can be mathemati¬ 
cally proven to work in terms of asymptotic stochastic guarantee of success (cf. 
works of Vose on simple genetic algorithm [32] ) • 

Agent-oriented systems should also be asynchronous by nature, as they are 
inspired by social or biological systems. Over the last decade, our group has 
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worked on the design and development of decentralized evolutionary compu¬ 
tations [2] in the form of Evolutionary Multi-Agent Systems [3]. EMAS is 
a hybrid meta-heuristic which combines multi-agent systems with evolutionary 
algorithms. A dedicated mathematical formalism, based on Markov chains (sim¬ 
ilar to Vose’s approach) was constructed and analysed [1], showing that EMAS 
may be also treated as a general-purpose optimization system. Besides that, a 
number of other formalisms along with dedicated frameworks implemented in 
different programming languages (like Java, Scala or Python) were developed 
(see, e.g. P[29l[7j). 

The concept of hybridization of agent-based systems with evolutionary tech¬ 
niques can be implemented in different ways, especially with regard to asyn¬ 
chronicity and concurrency, as well as distribution and parallelism. There is a 
number of popular agent-oriented frameworks which offer asynchronously com¬ 
municating agents (such as Jadex [27], Jade [1] or MadKit[l5]l. However, they 
all share similar properties, such as heavyweight agents, at least partial FIPA- 
compliancy (JADE) or a BDI model (Jadex). It is also common for each agent 
to be executed as a separate thread (e.g. in JADE). These traits are indeed 
appropriate to model flexible, coarse-grained, open systems. However, evidence 
suggest they are not best suited for closed systems with homogeneous agents 
nor for fine-grained concurrency with large numbers of lightweight agents, which 
are both common in biologically-inspired population-based computing systems 

m- 

Therefore, dedicated tools for the above-mentioned class of agent-based sys¬ 
tems have been constructed over the last 15 years. One of the successful im¬ 
plementations is the AgE platforiiQ which supports phase-model and hybrid 
concurrency features (parts of the system are concurrent and parts are imple¬ 
mented as sequential processes). The AgE platform also has other advantages, 
such as signihcant support for reuse and flexible conhguration management. 
Dedicated AgE implementations were constructed using Java, .NET and Python 
technologies. 

However, the renewed interest in functional programming and languages such 
as Erlang and Scala brought new possibilities in terms of concurrent program¬ 
ming support. In a recent paper EH, we proposed a promising new approach 
to these kinds of agent-based algorithms. We used Erlang lightweight processes 
to implement a fined grained multi-agent system and bring more asynchronicity 
into usually synchronously implemented agent actions and interactions. 

In this paper, we present a significant progress over the research presented in 
HB, by extending our experiments to a Scala-based implementation (in addition 
to the Erlang-based one) and comparing these two approaches. The previous 
Erlang asynchronous implementation showed a small but statistically significant 
improvement in terms of efficiency, compared to a synchronous version. The 
results for the new Scala implementation provide much stronger evidence for the 
superiority of a massively-concurrent EMAS implementation over a traditional. 
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synchronous one. 

In the next sections we present the current state-of-the-art and introduce 
concepts of evolutionary multi-agent systems. Then, we describe the implemen¬ 
tation of the synchronous and asynchronous versions of our algorithm, followed 
by our experimental settings and results. We end with a discussion of our results 
and conclude the paper with possible opportunities for future work. 

2. Large-scale Agent-based Systems 

The development of the software agent concepts and the theory of multi¬ 
agent systems took place in the last decades of the 20th century. At this point 
in time, the software engineering domain was strongly focused on the popu¬ 
larization of the object-oriented paradigm. As a result, the majority of agent 
systems and platforms for agent systems development was based on imperative 
languages with shared memory. This approach is in opposition to the assump¬ 
tions of agent systems, which are based on the concept of message passing, 
communication between autonomous execution threads and do not allow any 
explicit shared state. 

Implementations of message passing concurrency in object oriented tech¬ 
nologies resulted in significant limitations in both scale and performance of the 
developed solutions. The evaluation presented in [31] shows that the most pop¬ 
ular agent development platforms (Jade and Magentix) are limited to several 
thousands of simultaneous agents on a single computer. The limit was caused by 
the method used for implementing concurrently executing code of agents - each 
agent required a separate operating system thread. This situation inhibited the 
development of large scale multi-agent systems for long time. 

Current trends in the domain of programming languages development focus 
on the integration of concepts from different paradigms and on the development 
of new languages dedicated for particular purposes. The renewed interest in 
the functional paradigm seems very significant in the context of agent systems 
development. The agent system for human population simulation, presented 
in |12j . was implemented in Haskell. The authors emphasize that the source 
code was very short in respect to the complex functionality of the system. The 
discussion presented in m focuses on the language features of Haskell in the 
context of agent systems. The authors show the usefulness of algebraic data 
types, roles and sessions in implementation of multi-agent algorithms. 

The popularization of the functional paradigm concepts is mostly caused 
by difficulties in efficient usage of multi-core CPUs in languages implementing 
a shared-memory concurrency model. The need of synchronization of all the 
operations on shared memory effectively prevents the applications from scaling 
on higher numbers of cores. The issue does not exist in the message passing 
concurrency model, which allows massively concurrent applications to run effec¬ 
tively on parallel hardware architectures. This fact triggered the development 
of languages and runtime environments which efficiently implement the message 
passing concurrency model. There are currently two major technologies of this 
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kind being successfully used in industrial applications: Erlang and Scala with 
the Akka library. 

The concurrency model implemented by these technologies is based on the 
same assumptions as in the case of software agents. Therefore, these technolo¬ 
gies are a very good basis for large scale and high performance multi-agent 
systems. Recent example of a Scala-based implementation of a custom multi¬ 
agent architecture can be found in |22j . where the authors present a system 
capable of processing and storing a large amount of messages gathered from 
sensing different devices. 

Erlang technology has been found useful in this kind of applications much 
earlier. In 2003 the hrst agent development platform, called eXAT {erlang ex¬ 
perimental Agent Tool), has been presented in |11) . The goal of this platform 
was to test the feasibility of using functional programming languages as a devel¬ 
opment tool for FIPA-compliant agents. An agent in eXAT is an actor-based, 
independent entity composed of behaviours, which represent the functionality 
of an agent as a finite-state machine. Transitions between states are triggered 
by changes in the knowledge-base facts or by external messages. The original 
version of eXAT does not support agent migrations, however there is a version 
supporting this functionality [25] . 

eXAT platform overcomes the basic limitations of Java-based solutions. 
eXAT agents are based on Erlang lightweight processes, which can be created 
in millions on a single computer. Although the platform never became a main¬ 
stream tool, it should be noticed that it was the hrst environment which allowed 
to test the behaviour of large scale systems with truly parallel agents. 

Recent years brought different solutions based on Erlang technology, like the 
eJason system nnj. eJason is an Erlang implementation of Jason, which is a 
platform for the development of multi-agent systems developed in Java. The 
reason for rewriting the Jason platform in Erlang, pointed by the authors, are 
signihcant similarities between Jason agents and Erlang processes and the high 
performance and scalability of Erlang processes implementation. 

The ability to build and test massively-concurrent agent-based systems opens 
new possibilities of research in this domain. The algorithm presented in this 
paper is made possible by the high-performance implementations of a message¬ 
passing concurrency model offered by Erlang and Scala. 

3. Evolutionary Multi-Agent Systems (EMAS) 

Generally speaking, evolutionary algorithms are usually perceived as univer¬ 
sal optimization-capable metaheuristics (cf. theory of Vose |31j)- However, the 
classical designs of evolutionary algorithms (such as simple genetic algorithm 
m, evolution strategies etc. |30) 1 assume important simplihcations of the un¬ 
derlying biological phenomena. Such simplification mainly consists in avoiding 
direct implementation of such phenomena observed in real-world biological sys¬ 
tems, as dynamically changing environmental conditions, a dependency on mul¬ 
tiple criteria, the co-evolution of species, the evolution of the genotype-fenotype 
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mapping, the assumption of neither global knowledge nor generational synchro¬ 
nization. 

Of course, it does not mean that they are wrong per-se, as they have clearly 
proven themselves in solving difficult problems. However, there is still room for 
improvement, and the No Free Lunch Theorem |33j reminds us that the search 
for new optimization techniques will always be necessary. 

One of the important drawback of classical evolutionary techniques is that 
they work on a number of data structures (populations) and repeat in cycles 
(generations) the same process of selecting parents and producing offspring 
using variation operators. Such an approach makes it difficult to implement 
large-scale, parallel implementations of evolutionary algorithms. Only trivial 
approaches to the parallelization of such algorithms were proposed (e.g. the 
master-slave model or parallel evolutionary algorithm i)- 

For the over 10 years, our group tried to overcome some of these limita¬ 
tions by working on the idea of decentralised evolutionary computations [2], 
namely Evolutionary Multi-agent Systems (EMAS) [S]. EMAS is a hybrid meta¬ 
heuristics which combines multi-agent systems with evolutionary algorithms. 
The basic idea of EMAS consists in evolving a population of agents (containing 
the potential solutions to the problem in the form of genotypes). The agents 
are capable of doing different actions, communicating among themselves and 
with the environment, in order to find the optimal solution of the optimization 
problem. 

According to classic definitions (cf. e.g., [T7]) of a multi-agent system, there 
should not be global knowledge shared by all of agents. They should remain 
autonomous without the need to create any central authorities. Therefore, evo¬ 
lutionary mechanisms such as selection needs to be decentralized, in contrast 
with traditional evolutionary algorithms. Using agent terminology, one can say 
that selective pressure is required to emerge from peer to peer interactions be¬ 
tween agents instead of being globally-driven. 

Thus, selection in EMAS is achieved by introducing a non-renewable re¬ 
source, called life-energy. Agents receive part of the energy when they are 
introduced in the system. They exchange energy based on the quality of their 
solution to the problem: worse agents move part of their energy to the better 
ones. The agents reaching certain energy threshold may reproduce, while the 
ones with low amount of energy die and are removed from the system. We show 
the principle of these operation in Fig. For more details on EMAS design 
refer to [2]. 

Up till now, different EMAS implementations were applied to different prob¬ 
lems (global, multi-criteria and multi-modal optimization in continuous and dis¬ 
crete spaces), and the results clearly showed superior performance in comparison 
to classical approaches (see H). It is to note, that besides acclaimed bench¬ 
mark problems, as multi-modal functions [IHIISIISI: and discrete benchmarks 
with clear practical application, like Low Autocorrelation Binary Sequence or 
Golomb Ruler problem [HIIH] EMAS was also successfully applied to solving 
selected inverse problems [34l |28] leveraging its capability of quite low compu¬ 
tational cost evaluated as number of fitness function calls, compared to other 
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Figure 1: Structure and behaviour of EMAS operation. 


classic approaches. 

During the last years, we made also several approaches to construct efficient 
software targeted at variants of EMAS and at agent-based computing in general. 
We implemented dedicated tools in order to prepare fully-fledged frameworks 
convenient for EMAS computing (and several other purposes, as agent-based 
simulation). First, we focused on implementing decentralized agent behaviour, 
and the outcome were several fully synchronous versions, resulting in the imple¬ 
mentation of the AgE platfornE In this implementation we applied a phase- 
model of simulation, efficiently implementing such EMAS aspects as decentral¬ 
ized selection. We also supported the user with different component-oriented 
utilities, increasing reuse possibilities and allowing flexible configuration of the 
computing system. 

In a recent paper EH, we have presented a promising new approach to these 
kinds of algorithms. We used Erlang lightweight processes to implement a fine¬ 
grained multi-agent system. Agents are fully asynchronous and autonomous in 
fulfilling their goals, such as exchanging resources with others, reproducing or 
being removed from the system. Agents are able to coordinate their behaviour 
with the use of mediating entities called meeting arenas. 

This approach brings us closer to the biological origins of evolutionary al¬ 
gorithms by removing artificial generations imposed by step-based implementa¬ 
tions. We show the concept of generation-oriented and generation-free comput- 
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ing in Fig. 



a) b) c) 


Figure 2: Concept of generation-oriented and generation-free evolutionary computing. 


The first case (o) shows the classical approach, consisting in transforming a 
population of individuals by applying a stochastic state transition function. This 
function is usually composed of some predefined operators, such as selection, 
crossover and mutation. The second case (6) shows the EMAS approach (as 
it was realized in AgE platform), where different transformation functions are 
applied as the results of agent actions. This model still assumes the existence 
of generations, but on a technical level, as a result of a step-based simulation. 
In fact, both above cases use discrete-time based simulation. 

In contrast, the third case (c) is a nearly continuous-time simulation (if we 
disregard the discrete nature of the machine itself). In this model, all agents 
may initiate actions at any possible time. The process scheduler makes sure 
they are given computing resources when they need them. 

4. Massively-concurrent EMAS implementation 

The system presented in this work has been implemented in Erlang and 
Scala, as the lightweight concurrency model provided by these technologies is 
well suited for creating large-scale multi-agent systems. The implementation 
focuses on comparing different computational models in terms of their features 
and efficiency. The rest of this section describes the algorithms used in our 
evolutionary multi-agent system, the model of agents interactions and different 
implementations. 

4.I. Principle of system operation 

Every agent in the system is characterized by a vector of real values rep¬ 
resenting potential solution to the optimization problem. The vector is used 
for calculating the corresponding fitness value. The process of calculating the 
fitness value for a given solution is the most expensive operation. It is executed 
each time a new solution is generated in the system. 

Emergent selective pressure is achieved by giving agents a piece of non¬ 
renewable resource, called energy [2]. An initial amount of energy is given to a 
newly created agent by its parents. If the energy of two agents is below a required 
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threshold, they fight by comparing their fitness value - the better agent takes 
energy from the worse one. If an agent looses all its energy, it is removed from 
the system. Agents with enough energy reproduce and yield new agents. The 
genotype of the children is derived from their parents using variation operators. 
The number of agents may vary over time, however the system remains stable 
as the total energy remains constant. 

As in the case of other evolutionary algorithms, the population of agents can 
be split into separated sub-populations. This approach helps preserving popula¬ 
tion diversity by introducing allopatric speciation and can also simplify parallel 
execution of the algorithm. In our case the sub-populations are called islands. 
Information can be exchanged between the islands through agent migrations. 

4-2. Arenas 

An efficient implementation of meetings between agents is crucial for the 
overall performance of the algorithm. The meetings model has a significant 
impact on the properties of the algorithm, on its computational efficiency and 
on its potential for parallel execution. 

A general and simple way to perform meetings is to shuffle the list of agents 
and then process pairs of agents sequentially or in parallel. However, this ap¬ 
proach has several limitations: 

• The whole population must be collected in order to shuffle agents and form 
the pairs. This approach is inappropriate in an algorithm which should 
be decentralized by nature. 

• Agents willing to perform different actions can be grouped together - all 
combinations of possible behaviours must be handled. 

In our previous work |20], a different approach was proposed. We proposed 
to group agents willing to perform the same action in dedicated meeting arenas, 
following the Mediator design pattern. Every agent enters a selected arena 
depending on its amount of energy. Arenas split incoming agents into groups 
and trigger the actual meetings (see Fig. [^. Each kind of agent behaviour is 
represented by a separate arena. 



Agents 


Figure 3: Meeting arenas group similar agents and coordinate meetings between them. 
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Therefore, the dynamics of the multi-agent system are fully defined by func¬ 
tions. The first function represents agent behaviour, which chooses an arena 
for each agent. The second function represents the meeting operation which is 
applied in every arena. 

This approach is similar to the MapReduce model, where arena selection 
corresponds to mapping and meeting logic to reduce operation. The pattern is 
very flexible, as it can be implemented in both a centralized and synchronous 
way or a decentralized and asynchronous one. 

4-3. Sequential implementation 

The sequential version of the presented multi-agent system is implemented as 
a discrete event simulation. In each step the behaviour function (see Listing]^ 
divides the population of agents into groups corresponding to available arenas. 


1 

def beh 

a,viour(a: Agent) = a. energy match { 

2 

0 => 

death 

3 

X i f 

X > 10 => reproduction 

4 

X => 

fight 

5 

} 



Listing 1: In every step, agents choose an arena based on their current state 


Agents are first grouped according to their chosen arena and then a meeting 
function is applied on each such partition of the population. The partitions can 
be further subdivided into pairs of meeting agents and processed by applying a 
different meeting function which depends on the type of the arena (see Listing 
[^. Every meeting results in a group of agents. The group can contain some new 
agents created as a result of reproduction and some with their state changed 
(e.g. by transferring energy). Some agents may be removed from the group if 
their energy equals 0. Resulting groups are merged in order to form the new 
population, which is randomly shuffled before the next step. 

If several islands are considered, each is represented as separate lists of 
agents. Migration between islands is performed at the end of each step by 
moving some agents between lists. 

4-4- Hybrid implementation 

The introduction of coarse-grained concurrency in such a multi-agent system 
is rather straightforward. In our second implementation every island is assigned 
to a separate Erlang process/Scala actor responsible for executing the loop of the 
sequential algorithm described above. Islands communicate through message¬ 
passing, no other synchronization is needed. 

The most significant difference regards agent migration. The behaviour func¬ 
tion from Listingj^is modified by adding a migration action which is chosen with 
some fixed, low probability. The migration process is performed by a dedicated 
migration arena present on every island. 
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1 def meeting (arena: Arena, agents: Seq[Agent]) = 

2 arena match { 

3 death => 

4 Seq . empty [ Agent ] 

5 reproduction => 

6 agents 

7 . shuffle 

8 .grouped(2) 

9 . flatMap (doReproduce) 

10 fight => 

11 agents 

12 . shuffle 

13 .grouped(2) 

14 . flatMap ( doFight) 

15 } _ 

Listing 2: Depending on the type of the arena, a different meeting happens, which transforms 
the incoming subpopulation of agents. The death arena simply return an empty sequence. 
Other arenas shuffle incoming agents, group them into pairs and apply a binary operator on 
every pair, concatenating results. 


The migration arena removes agent from the local population and forwards 
it agent to a selected island chosen according to some topology and migration 
strategy. In every step the processes responsible for executing islands loop 
incorporates the incoming agents into their population. 

4-5. Concurrent implementation 

The first two implementation presented so far do not require massively con¬ 
current execution, as the agents are represented as data structures processed 
sequentially by islands and arenas. Such approach does not reflect the auton¬ 
omy of entities in the population and the true dynamics of relation between the 
entities, as each agent is forced to perform exactly one operation in each step 
of the algorithm. 

In order to achieve asynchronous behaviours of agents in the population, 
every agent and every arena has been implemented as a separate process/actor 
which communicates with the outside world only through message passing. The 
algorithm becomes fully asynchronous, as every agent acts at its own pace and 
there is no population-wide step. Meeting arenas are especially useful in this 
implementation, as they greatly simplify communication protocols. 

The algorithm of each agent is relatively simple. Depending on its current 
energy, every agent selects the action to perform and sends a message to the 
appropriate arena. Afterwards it waits for a message with the results of the 
meeting. 

As soon as enough agents gather in an arena, a meeting is triggered. As a 
result, new agents may be created and existing agents may be killed or replied 
with a message containing their new state. 
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Islands are logically defined as distinct sets of arenas. Each agent knows the 
addresses of all the arenas defining a single island, therefore it can only meet 
with other agents sharing the same set of arenas. Fights and reproductions 
arenas behave just as described in the previous version of the algorithm. 

This migration process is greatly simplified in this version. Migrating an 
agent simply means changing the arenas it meets on. Migration arenas choose 
an island according to specified topology and send the addresses of the corre¬ 
sponding arenas back to the agent. The agent updates the set of arenas available 
to itself and resumes its behaviour. As it will now be able to meet with a dif¬ 
ferent set of agents, it has indeed migrated. 

The implementations in Erlang and Scala are relatively similar, as both real¬ 
ize exactly the same algorithms. In case of first two approaches (sequential and 
hybrid) no differences in behaviour of the implemented system was expected. 
On the other hand the execution of massively concurrent version can be sig¬ 
nificantly dependent on scheduling mechanisms implemented by the underlying 
process management system. The time of activities performed by each agent 
depends on the provided CPU access. 

The details of process scheduling mechanisms implemented by Erlang and 
Scala (Akka) are slightly different. Erlang was designed as a soft real-time 
platform, which means that each a process can be preempted in every moment in 
time, and none can claim more computational time than the others. The model 
implemented by Scala is more suited for reactive, message-driven programming, 
where a process can use CPU until it finished processing current messages. 
These tiny differences can influence the overall performance of the implemented 
systems but also can result in different behaviour of the populations. 

Another difference is memory management. In Erlang, every process owns a 
separate stack and the content of messages needs to be copied between process 
memories. Scala uses the Java Memory Model and adds a message-passing layer 
on top of shared memory. The Akka actor library follows the principle ’’going 
from remote to local by way of optimization”, therefore communication within 
a single Java Virtual Machine is very memory efficient, as immutable messages 
are safe to be shared between the sender and receiver. 

5. Methodology and Results 

We used our multi-agent system to minimize the Rastrigin function, a com¬ 
mon benchmarking function used to compare evolutionary algorithms. This 
function is highly multimodal with many local minima and one global mini¬ 
mum equal 0 at x = 0. We used a problem size (the dimension of the function) 
equal to 100, in a domain equal to the hypercube [—50, 50]^°°. 

The simulations were run on Intel Xeon X5650 nodes provided by the Pl- 
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Grief] infrastructure at the ACC Cyfronet AGlf] We used up to 12 cores and 
1 GB of memory. 

We tested the alternative approaches described in the previous section, im¬ 
plemented in both Erlang ans Scala. The experiments for hybrid and concurrent 
models were run on 1,2,4,8 and 12 cores. A sequential version in each language 
was also run on 2 cores (the second core being used for logging an management) 
- more cores did not improve the results. 

Every experiment lasted 15 minutes and was repeated 30 times in order to 
obtain statistically significant results. The results further below are averaged 
over these 30 runs. 

The hybrid model does not benefit from a number of cores higher than the 
number of islands. As we had 12 cores at our disposal, we used 12 islands 
in every experiment. Migration destinations were chosen at random in a fully 
connected topology. 

Results. We examined our models under two criteria: how well the algorithm 
works and how fast the meeting mechanism is. 

We assessed the quality of the algorithm by recording: the fitness of the best 
solution found so far at any given time on any island (see Eig. [^. 

We estimated the speed of the models by counting the amount of agent meet¬ 
ings performed in a unit of time. These numbers appeared to be proportionally 
related across different arenas. Therefore, we only consider below the amount 
of reproductions per second (see Eig. [^. The number of reproductions may 
depend not only on the implementation and number of cores but also on the al¬ 
gorithm itself. Therefore, it is a useful metric of speed when comparing the same 
model but with different implementations and numbers of cores. However, the 
number of reproductions relates to the number of fitness function evaluations, 
it is also a metric of efficiency between the alternate models. 

Discussion. The results of the optimisation experiments are shown in Fig. 
Both models in both implementations improve when cores are added. The 
sequential versions in both Erlang and Scala behaved just like the hybrid models 
with 1 cores, minus some small communication overhead, so these results are 
omitted further on. 

The results for the Erlang and Scala implementations are similar in the 
early stages of experiments. In the later stages, the Erlang version becomes 
much slower than the Scala one. However, a close-up on the Erlang results 
reveals that its characteristics are also similar to the Scala results, but over a 
larger timespan. 

The Scala results show that the hybrid version is initially faster. However, 
the concurrent model takes over at some point and becomes much more effective 
than the hybrid model in the later stages. In fact, in the case of 12 cores, 100% 
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of the experiments with the concurrent model found the global optimum by the 
12th minute of the experiments. 

Another difference between the models is the number of reproduction hap¬ 
pening every second (Fig. [^. In the case of the Erlang implementation, these 
numbers increase nearly linearly with the increase of nodes. 

In the Scala implementation, the concurrent version also scales linearly, but 
the hybrid version drastically improves when the number of cores equals the 
number of islands. Looking at it another way, the performance of the Scala 
hybrid model drastically declines when there is less cores than islands, which 
is not the case of the Erlang implementation. In that regard. Erlang real-time 
scheduling prove to be more efficient than naive JVM threading (thread per 
island). 

In both implementations the number of reproduction per second is signifi¬ 
cantly smaller in the case of the concurrent version. The interpretation can be 
twofold: on one hand, they may indicate that the concurrent implementation is 
slow at processing agent meetings. On the other hand, the number of reproduc¬ 
tions reflect the number of fitness evaluations. As the concurrent version still 
achieves better results, but with fewer fitness evaluations, it can be considered 
more efficient. 

This observation becomes all the more evident when plotting the best fitness 
over the number of total reproductions instead of over time. Fig. [^confirms 
that the concurrent and hybrid models are in fact two different algorithms. 
Increasing the number of cores simply increases the number of reproductions 
per second and therefore reduces the time to reach a given value. However, the 
concurrent version needs a much lower number of reproductions, and therefore 
less function evaluations, to reach a given value. 

This difference in dynamics could be explained in the following way: in the 
hybrid version, agents in the population are effectively synchronized, in the 
sense that all fights and all reproductions in a step need to end for any agent 
to move on. In contrast, in the concurrent version fights happen independently 
of reproduction and the population evolves in a much more continuous way. 
Information spreads faster in the population and the solution can be found with 
fewer generations. 

Therefore, as the concurrent version needs less function evaluation, we con¬ 
jecture that it should perform even better compared to the hybrid one when 
faced with real-life problems, where the computation of the fitness function 
itself can take much time. 

The difference in performance between the Erlang and Scala versions can 
have several causes. First, the Erlang VM is usually less efficient than the 
Java VM when it comes to raw arithmetic. Both languages use 64 bit floating 
point numbers, though, so problems with numerical precision can be ruled out. 
Second, both languages strongly differ in memory management. In Erlang, every 
process owns a separate stack. With a few exceptions, all messages need to be 
copied from the memory of one process to another, even if the data is immutable 
and could be sent by reference. In contrast, Scala and Akka are based on the 
shared Java Memory Model [23], but offer different concurrency primitives in 
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order to use message passing. Therefore, immutable data can be transferred 
within a VM as fast as in Java and as safely as in Erlang. All in all, the 
Scala implementation incurs less overhead and the differences in characteristics 
between the algorithms are amplified. However, it is the insight from designing 
the algorithm for Erlang first that led us to that efficient implementation in 
Scala. 

6. Conclusions 

The massively-concurrent implementation of a evolutionary multi-agent sys¬ 
tem presented in this paper gives very promising results in terms of scalability 
and efficiency. Its asynchronous nature allows to better imitate the mechanisms 
observed in biological evolution, going beyond the classical approach of discrete 
generations and synchronous population changes. 

We applied this algorithm to a popular optimization benchmark. The results 
of our experiments indicate that when many agents are involved, the concurrent 
model is significantly more efhcient in terms of approaching the optimum versus 
number of fitness function evaluations. This result shows that this technique is 
very promising when the complexity of fitness function is high (e.g. in the case 
of solving inverse problems). 

The key to achieve an efficient implementation was using Erlang and Scala 
technologies, in particular their features like lightweight processes and fast mes¬ 
sage passing concurrency. The Scala version appears to be more efficient, mainly 
because of a better memory management of the underlying library. 

A further development of this method on modern multicore supercomputers 
[TB] seems a promising direction of research. Broader tests will also be performed 
in multicore systems consisting of a higher number of cores than examined here. 
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Figure 4: Best fitness ever over time, depending on the model and the number of cores. Stripes 
indicate 95% confidence intervals. 10“^® constant has been added to the fitness values to 
visualize the global optimum. 
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Figure 5: The relation of the number of reproductions per second to the number of cores, for 
each model and implementation, along with 95% confidence intervals. 
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Figure 6: Best fitness ever over the total number of reproductions. Confidence intervals are 
the same as in Fig. [^and have been omitted for clarity. 


21 
























